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ABSTRACT 

This study examined and assessed teachers 1 and 
students 1 use of and reactions to the Heartwood Program, a 
multicultural, literature-basec* curriculum designed to promote 
elementary students 1 ethical understanding of courage, loyalty, 
justice, respect, hope, honesty- and love. The Pennsylvania 
Department of Education Division of Federal Programs (Chapter 1) 
supported a field test and evaluation of the Heartwood Program in 16 
school districts across the state. In 15 of the districts, typically 
a Chapter 1 pull-out teacher was selected to participate in the field 
test; in the remaining district, the program was offered to all 
teachers in an elementary school. Information on teacher training, 
program materials, classroom use of the program, and program outcomes 
was collected. Results indicated that: (1) almost all teachers 
emphasized the importance and need for this type of program in their 
schools; (2) the program was teacher-friendly; (3) program materials 
should be reviewed to strengthen the sequencing of stories and 
activities, the coverage and portrayal of some minority cultures, and 
the level of difficulty of story lines for primary grade students; 
and (4) the appropriate placement of the program in a school's 
instructional program needs additional thought. (A list of interview 
questions is attached.) (RS) 
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Evaluation of Heartwood Program 
Executive Summary 

Joan L. Buttram, Janice Kruse, and Jennifer Sidler 
Research for Better Schools 

The Heartwood Program is a multicultural, literature-based curriculum 
designed to promote elementary students' ethical understanding of seven 
attributes: courage, loyalty, justice, respect, hope, honesty, and love. In 
the program, an adult reads a story to a group of students, discusses its 
contents and their students' reactions to it, and helps them complete related 
classroom activities. 

During the 1991-91 school year, the Pennsylvania Department of Education 
(PDE) Division of Federal Programs (Chapter 1) supported a field test and 
evaluation of the Heartwood Program in 16 school districts across the state. 
In 15 of the districts, typically a Chapter 1 pull-out teacher was selected to 
participate in the field test; in the remaining district, the program was 
offered to all teachers in an elementary school. 

The purpose of this evaluation was to examine and assess teachers' and 
students' use of and reactions to the Heartwood Program. RBS vis&^d field 
test teachers to collect information on teacher training, program materials, 
classroom us^^f the program, and program outcomes. The major findings of the 
evaluation are summarized below. 

• Almost all teachers emphasized the importance and need for this type 
of program in their schools. Teachers felt strongly that students 
were lacking in their ethical development, and that this program 
provided a constructive strategy for initiating discussions with 
students in this critical area. 

• The Heartwood Program is teacher-friendly. With very little 
training, teachers can learn the basic elements of the program. 
However, more time should be devoted to demonstrations of how to use 
the program materials to promote students' in-depth discussion and 
understanding of the seven attributes. 

• The program materials should be reviewed to strengthen: the 
sequencing of stories and activites from year to year; the coverage 
and portrayal of some minority cultures, especially Hispanics and 
African-American; and the level of difficulty of story lines for 
primary grade students. 

• The appropriate placement of the Heartwood Program in a school's 
instructional program needs additional thought. In the current field 
test, both Chapter 1 pull-out classrooms and self-contained 
classrooms were included. The results generally indicated that 
teachers felt the program worked best when they were able to use the 
program on a regular, more prolonged, and interdisciplinary basis. 
Pull-out programs where teacher-student interactions were extremely 
limited and outcomes explicitly defined (i.e., improvement of reading 
scores) were probably not the ideal situation. 
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Evaluation of Heartwood Program 
Final Report 



Joan L. Buttram, Janice Kruse, and Jennifer Sidler 
Research for Better Schools 



The Heartwood Program is a multicultural, literature-based curriculum 
that uses classic children's stories from around the world to present 
universal values to students in kindergarten through grade six. The program 
promotes ethical understanding by exposing the students to seven core 
attributes: courage, loyalty, justice, respect, hope, honesty, and love. In 
the program, an adult reads a story to a group of students, discusses its 
contents and their students' reactions to it, and helps them complete related 
classroom activities. Each Heartwood Program kit consists of a teacher's 
guide, the 21 books (three per attribute), an activity card for each book that 
suggests discussion questions and follow-up activities, a world map, and flags 
of the countries represented by the stories. 

During the 1991-91 school year,' the Pennsylvania Department of Education 
(PDE) Division of Federal Programs (Chapter 1) supported the field test of the 
Heartwood Program in 16 school districts across the state. These districts 
represented urban, rural, and suburban areas with primarily white, minority, 
or mixed student populations. In 15 of the districts, a single teacher was 
selected by the district Chapter 1 coordinator to participate in the field 
test; in most cases, this individual was a Chapter 1 teacher who worked with 
students as part of a pull-out reading program. In the remaining district 
(School District of Philadelphia), the program was offered to all teachers in 
an elementary school; most of the teachers were assigned to self-contained 
classrooms* The teachers from the first 15 districts attended a half-day 
training program in Lancaster, PA in mid-November; the Philadelphia teachers 
received their training in a half-day session at their school in mid-January. 
Following the training, teachers were encouraged to use the Heartwood Program 
in their classrooms as they desired. 

Evaluation Design and Procedures 

Research for Better Schools (RBS) was contracted to evaluate the 
Heartwood Program by PDE. The purpose of the evaluation during the 1991-92 
school year was to examine and assess teachers' and students' use and 
reactions to the Heartwood Program. Given the fairly short period of time 
allowed for the field test this school year, no formal assessment was 
conducted of the program's impacts on students (e.g., ethical understanding). 
RBS instead decided to focus the evaluation on collecting descriptive 
information on the program's use. This information was seen as useful to the 
Heartwood developers as they revise and modify the program's materials and to 
PDE as it considers future support of the program. 

In order to carry out this evaluation, RBS first met with several of the 
Heartwood Program developers and the head of the PDE Division of Federal 
Programs. At these meetings, the program's purpose, components and use, 
training and support provided to teachers, and evaluation objectives were 
discussed. RBS also obtained and reviewed a Heartwood Program kit. RBS 
constructed a loosely-structured teacher interview (see Appendix A) to guide 
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the collection of information on teacher training, program materials, 
classroom use of the program, and program outcomes at each of the Heartwood 
field test districts. This instrument was shared with the Heartwood Program 
developers for their review and comment. RBS also expected to observe 
individual teachers conducting Heartwood lessons in order to describe the 
program's use with students and their reactions to it. No formal observation 
scale was developed to gather this information. Instead, RBS planned to 
record the teacher's and students' actions and affect. 

RBS contacted by telephone and arranged visits with the 15 non- 
Philadelphia field test teachers with two exceptions. RBS was unable to 
arrange a visit with one teacher because of scheduling conflicts and the other 
teacher was unwilling to participate in the evaluation. During the 13 
completed visits, the teacher was interviewed about the Heartwood program and 
a Heartwood lesson was observed, using the instrument and procedures described 
above. These visits typically lasted about 90-120 minutes. RBS was unable to 
conduct an observation with two of the teachers because they had either 
discontinued or never used the program. 

In the elementary school in the School District of Philadelphia, 
teachers were asked to sign up for an interview and/or observation by RBS in 
late May. Eight teachers volunteered to be interviewed and one to be 
observed. Six of the eight interviewed teachers reported reading some of the 
storybooks as part of their literature or social studies programs during the 
spring, but seldom used any of the other program materials; the other two 
reported that they were not using the program. All of the Philadelphia 
teachers indicated that their classroom routines were well-established b^~ the 
mid-January training date and so it was difficult to incorporate significant 
portions of the Heartwood program into their instructional program so late in 
the school year. 

RBS staff members recorded interview comments and observation notes 
after completing each visit. The interview and observation data were 
aggregated and analyzed once all of the data collection was completed. 
Because the Heartwood Program received such limited use in the Philadelphia 
classrooms, the interview and observation data were of limited use in these 
analyses. Nevertheless, their feedback was incorporated where appropriate and 
relevant . 

Evaluation Findings 

RBS' findings are summarized below. They are organized according to the 
four interview categories listed above -- teacher training, program materials, 
classroom use of materials, and program outcomes. 

Teacher training . The majority of teachers remembered the Heartwood 
training with enthusiasm. The trainer was described by many as "committed," 
"compassionate," "enthusiastic," and "genuine." They vaguely recalled being 
told about the origins and theory behind the Heartwood Program and its 
different components; few were able to remember many concrete details of the 
training that had occurred several months past. In spite of the comparatively 
short time allotted for training, few felt that additional assistance was 
needed. Those who did wish for more training suggested that additional 
guidance was needed on fostering ethical discussions, that is, how to plan 




activities that would emphasize and reinforce ethical understanding on the 
part of students. Others asked for sample lessons. About half of the 
teachers indicated that a follow-up session where they could share experiences 
and exchange ideas would be helpful; the other half did not suggest any 
follow-up assistance . 

Program materials . All of the field test teachers were able to list and 
point to all of the Heartwood materials in their classrooms. The Heartwood 
Program materials were highly commended by the field test teachers. One 
Chapter 1 teacher declared that the developers deserved a "gold star" for 
organizing the materials and making the program so easy for teachers to use; 
this sentiment was echoed by many others who spoke glowingly of the 
flexibility, adaptability, and organization of the Heartwood materials. 

The children's stories were the most praised part of the Heartwood 
Program. All of the teachers complimented the selection of the books for 
their attention to story and illustration. Although most of the teachers felt 
that the inclusion of ethical and/or multicultural themes was well done, some 
teachers expressed concern. This was especially the case in the Philadelphia 
school. For example, a few teachers noted that the Heartwood-identif ied 
attribute for a particular story was not always the most prominent attribute. 
Teachers with a large percentage of Spanish-speaking students bemoaned the 
absence of stories about Hispanic cultures. Teachers of classes with African- 
American students were split on their assessments of the books. Some praised 
the inclusion of books with African-American characters or themer; others felt 
that African-Americans were not portrayed positively or were not the central 
character in tfre story (e.g., Peewee Reese was featured over Jackie Robinson 
in Teammates , Peg Leg overshadowed the slaves escaping tht South in Follow the 
Drinking Gourd ) . In addition, many of the teachers felt that some of the 
stories were too difficult for younger students (below third grade) or those 
with extremely limited backgrounds and suggested adding books to the 
collection that would be easier for these students to understand. 

The activity cards containing discussion questions and follow-up 
activities were seen as "terrific starting points" by most of the staff. One 
beginning teacher felt that these cards "helped her to use the program. " 
Other more experienced teachers reported mixed use of the activity cards; some 
glanced at them quickly but often developed their own activities. Teachers 
with primary grade rtudents reported that the suggested activities often were 
inappropriate for their young students. 

Classroom use of program . The use of the Heartwood Program differed 
significantly in Philadelphia versus non-Philadelphia classrooms. In 
Philadelphia, the program's use was significantly constricted. Six of the 
eight classroom teachers reported in their interviews that they read Heartwood 
stories to their students either weekly or biweekly, while one reported 
monthly readings, and another one indicated that she was not using the 
program. Only two of the teachers used the discussion questions and follow-up 
activities to any extent; the other teachers felt that the questions and 
activities were too time consuming or did not easily fit into their already 
established classroom routines. As a result of their limited use of the 
program, it was not feasible to gather meaningful classroom use data and so 
most of the data reported in this section are based on the experiences of the 
non-Philadelphia teachers. 



Twelve of the 13 non-Philadelphia teachers had used the Heartwood 
program in the months intervening between training and RBS' visit. Seven of 
these teachers were using the Heartwood Program in their pull-out Chapter 1 
reading classes in grades 1-5, two were using it in a Chapter 1-regular 
education teaming effort in reading in grades 4 and 9, one was using it in a 
self-contained reading class in combined grades 6 and 7, one was using it in a 
self-contained, Chapter 1 second grade classroom, and one was using it in a 
self-contained first grade classroom. One Chapter 1 teacher decided not to 
use the program because she felt the materials were "too babyish" for her 
fifth grade students. Frequency of use in these classrooms varied 
considerably, from twice weekly to once a month, with most using it every 
second or third week. Many of the Chapter 1 teachers in pull-out assignments 
reported that they were not comfortable using the Heartwood Program more often 
because of their limited contact time with students (i.e., typically between 
30 and 45 minutes each week) and the pressure they felt to concentrate on 
improving reading skills. In addition, they felt that the activities required 
too much time for them to use along with the stories in their limited periods. 

Teachers reported that it took v^ry little time to develop a Heartwood 
lesson; about 20 minutes seemed to be the average. When delivered, the 
lessons typically involved a short orientation to the story, usually about 5- 
10 minutes depending on the difficulty of the book and the age of the 
students; reading the story, usually 15-20 minutes; follow-up discussion, from 
5 to 15 minutes depending on the period of time available; and some follow-up 
activities. Teachers showed the most diversity in their inclusion of follow- 
up activities, many of which emphasized writing or art. Teachers' inclusion 
of follow-up activities was somewhat dependent on the classroom's grade level 
and pull-out or team/ self -contained status. Several teachers planned 
extensive follow-up activities that often required 3-4 hours of preparation on 
their part and several class periods for students to complete. 

RBS saw a wide variety of activities during its observations, including 
teachers reading stories to students; teachers and students discussing story 
lines, characters, and the attributes; and students completing follow-up 
activities. Not surprising, teachers seemed very proficient at reading the 
stories and leading student discussions to check comprehension. In nine of 
the 12 observations (75.0 percent), teachers or students were engaged in 
discussions or other activities that touched on the unlerlying attribute. 
However, in many of these observations, the discussions or activities were 
superficial and not directed at deepening students* understanding of the 
attribute. In the other three observations, teachers attended primarily to 
fairly narrow discussions of the story line or characters. 

Heartwood outcomes . Most of the teachers reported that insufficient 
time had elapsed to determine the program's impacts on students. Teachers 
generally felt that students were learning about the attributes, but were 
unable to document any specific effects. Several teacher related anecdotes 
about students' labeling other stories or characters as illustrating a 
particular attribute. One teacher reported that students in her pull-out 
class seemed to be more tolerant and understanding of each other. All cf the 
teachers stressed the importance of programs like Heartwood and the critical 
need for teachers to promote students' ethical understanding and behavior, but 
also underscored the difficulty of measuring student growth in this area. 
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Evaluation Findings 

Based on RBS' limited evaluation, the following conclusions were made 
about the Heartwood Program. 

• Almost all teachers emphasized the importance and need for this type 
of program in their schools. Teachers felt strongly that students 
were lacking in their ethical development, and that this program 
provided a constructive strategy for initiating discussions with 
students in this critical area. 

• The Heartwood Program is teacher-friendly. With very little 
training, teachers can learn the basic elements of the program. 
However, the training should be carefully examined to ensure that 
sufficient time is allotted for demonstrating how to use the stories, 
discussion questions, and follow-up activities to foster students' 
in-depth understanding of the seven attributes. 

• A careful review of the different stories should be carried out to 
make sure that all of the major racial and ethnic groups encountered 
by teachers are represented and presented positively. In addition, 
this review should examine the difficulty level of the stories to 
ensure that the story collection and activities are appropriate for 
primary grade students. 

• Some thought should be given to the sequencing of the Heartwood 
Program from year to year. If the program is to be used over the 
course of the entire elementary grades, then the number of stories 
and accompanying materials should be increased to accomodate repeated 
years of exposure. 

• Additional consideration should be given to the appropriate placement 
of the Heartwood Program in a school's instructional program; it may 
not be realistic to expect the program to function effectively in 
pull-out situations where instructional time is so tightly 
constrained . 
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Heartwood Interview Questions 

Teacher Training 

• What training do teachers receive to facilitate their use of the 
Heartwood curriculum? 

• What foilow-up assistance is available to help teachers use the Heartwood 
cur riculum? 

• What assistance is provided to teachers as part of the written Heartwood 
curriculum? 

• What other types of training activities or assistance would help 
teachers use the Heartwood curriculum? 

Curriculum Materials 

• What materials are included in the Heartwood curriculum for teachers to 
use with students? 

• Does the Heartwood curriculum appeal to the various cultures present in 
teachers' classrooms? Do any groups feel over- or under-represented? 
Explain, 

• Is there sufficient variety in the reading level, story lines, and 
cultures for the Heartwood stories to appeal to a broad cross-section of 
children? Explain . 

• Is there sufficient variety in the discussion questions, activities, and 
interdisciplinary ideas that accompany each story to appeal to a broad 
cross-section of children? Explain, 

• Are there sufficient instructions to help teachers read the stories and 
use the activity cards with students? Explain. What other items should 
be included? 

• What other assistance would be helpful to teachers in facilitating their 
use of the Heartwood curriculum with students? 

Classroom Use of Materials 

• How are teachers using the Heartwood curriculum in their classrooms? Do 
they follow the recommended lesson plans? What, if any, changes have 
they made in the recommended use? Explain, 

• How much time do teachers spend preparing to use the Heartwood curriculum 
in their classrooms? 

• How often do teachers use the Heartwood curriculum in their classrooms? 
As part of what disciplines or subject areas? 



• How do teachers react to the Heartwood curriculum? Are they comfortable 
using it7 Do they feel comfortable talking to students about these 
issues7 



# How do students react to the Heartwood curriculum? Are they eager to 
hear the stories and complete the activities? 

Heartwood Outcomes 

# What have students learned about the seven attributes? 

# Have students' reading scores changed during their participation in the 
Heartwood program7 

# Has the school and/or classroom climate changed during students' 
participation in the program7 



